Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bedrock client #213

Open
wants to merge 3 commits into
base: main
Choose a base branch
from

Conversation

khalilfiremind
Copy link

@khalilfiremind khalilfiremind commented Sep 15, 2024

AWS Bedrock Client has been added.

The user can access LLMs via single API:

  • AWS Titan
  • Claude
  • Cohere
  • LLama
  • Mistral
  • Jamba

The user can access AWS Bedrock visa profile (if SSO is set), or via secret key, access key,

or if the user assuming a role that have access to Bedrock (deploying on AWS lambda), then no need to pass anything, the client will have access automatically.

The client currently does not support asynchronous calls or streaming. These features may be added in future PRs.

Example of usage:

import os
from adalflow.core import Component, Generator, Prompt
from adalflow.components.model_client import BedrockAPIClient

PROFILE_NAME = os.getenv("PROFILE_NAME")
REGION_NAME = os.getenv("REGION_NAME",)

class SimpleQA(Component):
    def __init__(self):
        super().__init__()
        template = r"""<SYS>
        You are a helpful assistant.
        </SYS>
        User: {{input_str}}
        You:
        """
        self.generator = Generator(
            model_client=BedrockAPIClient(
                aws_profile_name=PROFILE_NAME, 
                aws_region_name=REGION_NAME), 
            model_kwargs={
                "modelId": "anthropic.claude-3-sonnet-20240229-v1:0",
                "inferenceConfig": {
                    "temperature": 0.8
                }
            }, template=template
        )

    def call(self, query):
        return self.generator({"input_str": query})


qa = SimpleQA()
answer = qa("What is LightRAG?")
print(answer)

I'm open to any feedback, notes, or suggestions on this PR.

Copy link

@chainyo chainyo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks great. I wish this will be soon merged, I would love to try AdalFlow on Bedrock right now.

Maybe the example should be added to the docs, no?

@khalilfiremind
Copy link
Author

@chainyo thanks man!!

yes sure, I can add docs

moreover, I want to add support for embedding models as next step!

@Hearsch-Jariwala
Copy link

@chainyo thanks man!!

yes sure, I can add docs

moreover, I want to add support for embedding models as next step!

@khalilfiremind Yeah this looks great, need this PR to go through! The use case I am trying to use it for is email attachments which can contain PDFs, docs, images, excel, etc...If you have a tutorial on a multimodal RAG for this that would be great to add using titan embeddings!

@khalilfiremind
Copy link
Author

@Hearsch-Jariwala thanks!

I will work on adding the embedding in the next few days! 🚀

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants